Late Fusion Approach for Multimodal Emotion Recognition Based on Convolutional and Graph Neural Networks

The current trends in automatic emotion recognition encompass the application of deep learning techniques, as, if applied to a multimodal approach, give the most promising results. The study presented in the paper follows this trend - the objective of the research is to propose a deep learning-based solution allowing to recognize emotions in circumplex model with performance metrics on a par with the ones achieved by competitive solutions. The observation channels used are physiological signals i.e. electrocardiography, electroencephalography and electroder- mal activity, while the applied technique is late fusion with Graph and Convolutional Neural Networks. The solution is validated for the AMIGOS dataset and the achieved results are com- parable to the baseline methods. While already satisfactory, the results still leave a place for further investigations

Ontology-Based Method for Analysis of Inconsistency Factors in Emotion Recognition

In the paper the problem of inconsistency in emotion recognition is approached. One of the existing challenges is the exploration of factors, which can influence the inconsistency. Therefore the aim of the paper is to present a method that allows capturing knowledge of what factors and what values of these factors influence the inconsistencies between recognized emotional states. The high-level, semi-automatic method allowing to recognize these factors is presented. The input of the method is the structured dataset and the output is the set of rules identifying when recognized emotional states are consistent or not. The presented method is validated for the dataset prepared for emotion recognition from face expressions using various methods

Automatic Emotion Recognition in Children with Autism: A Systematic Literature Review

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).The automatic emotion recognition domain brings new methods and technologies that might be used to enhance therapy of children with autism. The paper aims at the exploration of methods and tools used to recognize emotions in children. It presents a literature review study that was performed using a systematic approach and PRISMA methodology for reporting quantitative and qualitative results. Diverse observation channels and modalities are used in the analyzed studies, including facial expressions, prosody of speech, and physiological signals. Regarding representation models, the basic emotions are the most frequently recognized, especially happiness, fear, and sadness. Both single-channel and multichannel approaches are applied, with a preference for the first one. For multimodal recognition, early fusion was the most frequently applied. SVM and neural networks were the most popular for building classifiers. Qualitative analysis revealed important clues on participant group construction and the most common combinations of modalities and methods. All channels are reported to be prone to some disturbance, and as a result, information on a specific symptoms of emotions might be temporarily or permanently unavailable. The challenges of proper stimuli, labelling methods, and the creation of open datasets were also identified.Peer reviewedFinal Published versio

Clinical Outcomes of Extracranial Carotid Artery-Related Stroke Eligible for Mechanical Reperfusion on Top of Per-Guidelines Thrombolytic Therapy:Analysis from a 6-Month Consecutive Patient Sample in 2 Centers

BACKGROUND: Systemic intravenous thrombolysis and mechanical thrombectomy (MT) are guideline-recommended reperfusion therapies in large-vessel-occlusion ischemic stroke. However, for acute ischemic stroke of extracranial carotid artery origin (AIS-CA) there have been no specific trials, resulting in a data gap. MATERIAL/METHODS: We evaluated referral/treatment pathways, serial imaging, and neurologic 90-day outcomes in consecutive patients, presenting in a real-life series in 2 stroke centers over a period of 6 months, with AIS-CA eligible for emergency mechanical reperfusion (EMR) on top of thrombolysis as per guideline criteria. RESULTS: Of 30 EMR-eligible patients (33.3% in-window for thrombolysis and thrombolysed, 73.3% male, age 39-87 years, median Alberta Stroke Program Early Computed Tomography Score (ASPECTS) 10, pre-stroke mRS 0–1 in all, tandem lesions 26.7%), 20 (66.7%) were EMR-referred (60% – endovascular, 6.7% – surgery referrals). Only 40% received EMR, nearly exclusively in stroke centers with carotid artery stenting (CAS) expertise (100% eligible patient acceptance rate, 100% treatment delivery involving CAS±MT with culprit lesion sequestration using micronet-covered stents). The emergency surgery rate was 0%. Baseline clinical and imaging characteristics did not differ between EMR-treated and EMR-untreated patients. Ninety-day neurologic status was profoundly better in EMR-treated patients: mRS 0–2 (91.7% vs 0%; P<0.001); mRS 3–5 (8.3% vs 88.9%; P<0.001), mRS 6 (0% vs 11.1%; P<0.001). CONCLUSIONS: In a real-life AIS-CA setting, the referral rate of EMR-eligible patients for EMR was low, and the treatment rate was even lower. AIS-CA revascularization was delivered predominantly in stroke thrombectomy-capable cardioangiology centers, resulting in overwhelmingly superior patient outcome. Large vessel occlusion stroke referral and management pathways should involve centers with proximal-protected CAS expertise. AIS-CA, irrespective of any thrombolysis administration, is a hyperacute cerebral emergency and EMR-eligible patients should be immediately referred for mechanical reperfusion

Ontologie w Sieci Semantycznej

One of the major and most challenging tasks for modern Information Technology is development of methods aimed at automatically acquiring and processing knowledge stored in the biggest information repository the man has ever created – the Internet. This knowledge is of miscellaneous nature, mainly due to the fact that it is stored in many languages and in numerous formats with different levels of structur­ing. The “Semantic Web initiative” strives to achieve this goal by struc­turing the con­tents of Internet into publicly available and shared ontologies formulated in a com­monly accepted, machine readable format. In this paper we discuss problems of build­ing ontologies and using them throughout Semantic Web. Relevant topics are pre­sented in the broader context of knowledge representation methods. We also present an alternative approach based on processing textual Web contents, extracting seman­tics from them and creating a “general ontology” to be used to present knowledge to a user.Wraz z nastaniem ery Internetu i jego gwałtownym rozwojem, za­sad­niczym problemem dla współczesnej informatyki stała się automatyzacja pozys­kiwania olbrzymich zasobów wiedzy ludzkiej w nim zgromadzonych. Wiedza ta ma bardzo zróżnicowany charakter z uwagi na wielość formatów zapisu danych, a przede wszystkim z uwagi na różny stopień jej ustrukturalizowania. Jedną z najbardziej po­pu­larnych idei, dążących do systematycznego podejścia do pozyskiwania wiedzy z In­ter­netu stała się, tzw. „inicjatywa Semantic Web”. Zgodnie z tą ideą, wiedza ludzka powinna być strukturalizowana w formie ontologii publikowanych w Internecie, w po­w­szechnie zaakceptowanym, precyzyjnym i możliwym do przetwarzania przez komputery, formacie. W tym artykule bliżej prezentujemy problemy, związane z bu­do­wą ontologii i ich wykorzystaniem w ramach Semantic Web (Sieci Semantycznej). Problemy te przedstawiane są na szerszym tle różnych metod maszynowej reprezen­tacji wiedzy. Prezentujemy także podejście alternatywne, polegające na automatycz­nym tworzeniu „ontologii wszystkiego”, na podstawie analizy tekstu stron WWW w ję­zyku naturalnym

Ideologiczny i praktyczny model metaontologii

The development of Semantic Web initiative resulted in development of many languages for defining and manipulating knowledge. An important aspect of design of those languages is design of terminological statements. The paper presents the way the terminological statements are realized in KQL language (acronym for Knowledge Query Language) – an access language for RKaSeA knowledge management system.Rozwój inicjatywy Semantic Web spowodował rozwój różnych języków definiowania wiedzy i manipulowania nią. W ramach tych języków istotnym elementem jest zaprojektowanie rozkazów terminologicznych. W artykule przedstawiono sposób realizacji zapytań terminologicznych w języku KQL (Knowledge Query Language) – języku dostępu do systemu zarządzania wiedzą RKaSeA

Odwzorowania międzyontologiczne w algebrze konglomeratów

Modularization and integration of ontologies are domains that are recently of great interest among researchers in the domain of Semantic Web. In the paper we analyze the possibility of expressing two main branches of methods: mappings and links by means of s-module algebra and discuss possibility of using the algebra as a uniform medium of description of such methods.Modularyzacja i integracja ontologii to dziedziny, które w ostatnim okresie są obiektem intensywnego rozwoju. Rozwój nowych idei spowodował konieczność wprowadzenia ich systematyzacji i klasyfikacji. W niniejszym artykule przeanalizowano możliwość wyrażenia odwzorowań i złączeń za pomocą algebry konglomeratów oraz przedyskutowano możliwość wykorzystania roli algebry jako ujednoliconego medium opisu metod integracji i modularyzacji

Emotion Recognition from Physiological Channels Using Graph Neural Network

In recent years, a number of new research papers have emerged on the application of neural networks in affective computing. One of the newest trends observed is the utilization of graph neural networks (GNNs) to recognize emotions. The study presented in the paper follows this trend. Within the work, GraphSleepNet (a GNN for classifying the stages of sleep) was adjusted for emotion recognition and validated for this purpose. The key assumption of the validation was to analyze its correctness for the Circumplex model to further analyze the solution for emotion recognition in the Ekman modal. The novelty of this research is not only the utilization of a GNN network with GraphSleepNet architecture for emotion recognition, but also the analysis of the potential of emotion recognition based on differential entropy features in the Ekman model with a neutral state and a special focus on continuous emotion recognition during the performance of an activity The GNN was validated against the AMIGOS dataset. The research shows how the use of various modalities influences the correctness of the recognition of basic emotions and the neutral state. Moreover, the correctness of the recognition of basic emotions is validated for two configurations of the GNN. The results show numerous interesting observations for Ekman’s model while the accuracy of the Circumplex model is similar to the baseline methods

Język KQL jako realizacja idei języka SQL dla bazy wiedzy

As a part of the Semantic Web initiative there are being developed knowledge inference systems. Moreover, the languages for accessing such systems are also open issues. The paper proposes a new language for accessing modularized knowledge bases, whose features allows for its comprehensive usage in knowledge management systems in an analogous way as SQL language in relational database systems.W ramach inicjatywy Semantic Web rozwijane są systemy wnioskowania z wiedzy. Ciągle otwartym problemem są również języki dostępu do takich systemów. W artykule zaproponowano nowy język dostępu do zmodularyzowanych baz wiedzy, o cechach umożliwiających jego kompleksowe wykorzystanie w systemach zarządzania wiedzą, w sposób analogiczny do wykorzystania języka SQL w systemach relacyjnych baz danych